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(57) Abstract: Data is bidden in an infonnation 
signal by encoding payload in to relative positions 
and/or polarities of multiple (possibly equal) 
embedded watemiarks. The payload is retrieved by 
computing the correlation between the watefmark(s) 
and the signal as a function of the position. The 
distance between peaks of the cotrelation function 
represents die payload. In oider to precisely detect 
said distance, even if the peaks arc smeared, the 
correlation function in a window (W|) around one 
peak (pi) is compared with the correlation function 
in a similar window (W2) around another peak (P2). 
This is done for different positions of said windows. 
The distance (k) between the windows (37, 38) for 
which the correlation fiinctioos most resemble each 
other is the distance lepresenting the payload. 
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Detection of auxiliary data in an information signal. 
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FIELD OF THE INVENTION 

The invention relates to a method and arrangement for detecting pay load data 
encoded into relative positions and/or relative polarities of watermarks embedded in an 
information signal. 

5 

BACKGROUND OF THE INVENTION 

International Patent Application WO-A-99/45705 discloses a method of 
detecting multi-bit payload data in an information signal. The payload is encoded into the 
relative positions and polarities of multiple (possibly equal) embedded vsratermark patterns. 

1 0 The prior-art detection method comprises computing the correlation of the embedded 

v^ratermark patterns and the information signal as a function of the position of the v^atermark 
patterns with respect to the signal. This computing step yields a correlation function in which 
the positions and polarities of the embedded watermark patterns are represented by large 
positive or negative peaks. The relative distance between, and the signs of, the peaks 

1 5 represent the payload data. 

In the ideal case, the peaks of the correlation function resemble Dirac-pulses 
and their positions can be determined with sufficient precision. However, in many practical 
situations the positions and the polarities are not so well defined. This is particularly the case 
if the signal has been subjected to processing. Video processing, for example, causes the 

20 correlation peaks to be smeared. Peaks are broadened, so that the central position is no longer 
well defined. Even the sign of a peak may be incorrectly retrieved due to large negative side 
lobs. 

As a solution to this problem, WO-A-99/45705 teaches restriction of allowed 
peak positions to a sub-grid of all possible positions. Peak jitter is flien resolved by 
25 quantizing to the nearest sub-grid point. However, the payload capacity of the watermark is 
thereby reduced. Moreover, it does not solve the inversion of peak polarity. 
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OBJECT AND SUMMARY OF THE INVENTION 

It is an object of the invention to provide an improved method and 
arrangement for detecting the distance between multiple watermark patterns embedded in an 
information signal. It is also an object of the invention to provide an improved method and 
arrangement for detecting the relative polarities of said watermark patterns. 

To this end, the step of determining the relative positions of peaks comprises 
computing the resemblance of the correlation function in a window around one of said peaks 
to the correlation function in a window around another one of said peaks for a plurality of 
positions of at least one of said windows, and determining die positions of said windows for 
which said resemblance has an extreme value. 

The invention is based on the recognition that if peaks of a correlation function 
are smeared, they are all smeared in the same manner. The correlation functions (the 
waveforms) in the neigjxborhoods of peaks resemble each other to a large extent. The 
invention exploits this property by identifying the neighborhoods in which the correlation 
functions resemble each other most, and determining the distance between the identified 
neighborhoods rather than the distance between individual peaks. 

The same solution applies to the detection of the relative polarities of the 
peaks. In accordance with the invention, the step of determining the relative polarities of 
peaks comprises computing the resemblance of the correlation function in a v^dndow around 
one of said peaks to the correlation function in a window around another one of said peaks, 
and determining the sign of said resemblance value. If the sign is positive, the two peaks have 
the same polarity. If the sign is negative, the peaks have opposite polarities. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a schematic diagram of a watermark embedder to illustrate how 
payload data is embedded in a video image. 

Figs. 2 and 3 show diagrams to illustrate the operation of the watermark 
embedder which is shown in Fig. 1 . 

Fig. 4 shows a schematic diagram of an arrangement for detecting payload 
data in accordance with the invention. 

Figs. 5 A and 5B show two-dimensional correlation functions to generally 
illustrate the operation of the arrangement, which is shown in Fig. 4. 

Fig. 6 shows one-dimensional correlation functions to generally illustmte the 
operation of the arrangement, which is shovm in Fig. 4. 
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Fig. 7 shows one-dimensional correlation functions to illustrate one 
embodiment of the method in accordance with the invention. 

Figs. 8 and 9 show flow charts of operational steps which are carried out by 
embodiments of the arrangement in accordance with the invention. 

Fig. 10 shows a two-dimensional correlation peak pattern to illustrate a further 
embodiment of the method in accordance with the invention. 

DESCRIPTION OF EMBODIMENTS 

Fig. 1 shows a schematic diagram of a watermark embedder to illustrate how 
auxiliary data is embedded in a video image according to the teaching of International Patent 
Application WO-A-99/45705. The arrangement receives a motion video signal X and outputs 
a watermarked video signal Y. It comprises a payload encoder 10, a tiling circuit 11, and an 
adder 12. Fig. 2 illustrates the operation of the payload encoder 10. The encoder receives a 
limited set of uncorrelated "basic" watermark patterns (W1,W2). Each basic watermark is a 
given pattern of random noise luminance values, but for convenience it has been given 
human recognizable contents (a rectangle and a triangle, respectively) in this example. 

The embedded watermark pattern W is a combination of said basic patterns 
and/or cyclically shifted versions thereof. In this example, the encoder 10 generates W=W1- 
W2k, where W2ic is a cyclically shifted version of basic pattem W2. The signs and shift 
vectors (k) represent payload data K. To reduce complexity, the watermark pattem W has a 
relatively small size of MxM (e.g. 128x 128) pixels. As shown in Fig. 3, it is tiled over the 
larger image area by the tiling circuit 11. The same watermark WM is subsequently 
embedded in successive frames of a motion video signal. 

Fig. 4 shows a schematic diagram of the watermark detector in accordance 
with the invention. The watermark detector receives a possibly watermarked image Q. The 
image is partitioned (21) into blocks of size MxM (M=128) and all the blocks are stacked 
(22) in a buffer B of size MxM. This operation is known as folding. To detect v^ether or not 
the buffer B includes one or more versions of the watermarks Wl and W2, the buffer 
contents B and the respective watermark patterns are subjected to correlation. Computing the 
correlation comprises computing the inner product d=<q,w> of the information signal values 
q={qij} and the corresponding values w={wy}of the watermark pattem . For the two- 
dimensional MxM image, the inner product is: 

I M M 
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The watennark w is said to be present if the correlation d exceeds a predetermined threshold. 

The arrangement, which is shown in Fig. 4, uses the Fast Fourier Transform to 
simultaneously compute the correlation values d for all possible shifts of the basic watennark 
w with respect to the image, in accordance with the teachings of Intemational Patent 
Application WO-A-99/45707. This operation will be briefly summarized. Both the contents q 
of the buffer B and the basic watermark pattem w are subjected to a Fast Fourier Transform 
(FFT) in transforming circuits 23 and 24, respectively. These operations yield: 

q = FFT(q) and 

w = FFT(w), 

where q and w are sets of complex numbers. 

Computmg the correlation is similar to computing the convolution of q and the 
conjugate of w. In the transform domain, this corresponds to: 

d = q®conj(w) 

where the symbol ® denotes pointwise multiplication and conjQ denotes inverting the sign 
of the imaginary part of the argument. In Fig. 4, the conjugation of w is carried out by a 
conjugation circuit 25, and the pointwise multiplication is carried out by a multiplier 26. The 
set of correlation values d is now obtained by inverse Fourier transforming the result of said 
multiplication: 

d = IFFT(d) 

which is carried out by an inverse FFT circuit 27. 

Fig. 5 A shows a graph of correlation values if the basic watermark pattem Wl 
is applied to the detector. The correlation function exhibits a peak 31. The peak has a positive 
polarity and happens to be located at (0,0) in this example. Note that the absolute location of 
this peak may be different if the image Q has undergone processing such as translation or 
cropping. Fig. 5B shows the graph of correlation values if the basic watermark patteni W2 is 
applied to the detector. The correlation function now exhibits a negative peak 32 at (24,40). 
The relative position of this peak with respect to the position of peak 31 reveals the relative 
distance (in pixels) of W2k with respect to Wl, i.e. the shift vector k. 

A shift vector retrieving circuit 28 retrieves the shift vector k from the peak 
patterns in a manner to be described hereinafter. Finally, apayload decoder 29 decodes the 
embedded payload K from the vector k thus found and the polarities of the peaks. 

Fig. 6 shows the correlation functions in the one-dimensional domain for more 
convenience. The distance k between the peaks 31 and 32 can be precisely retrieved in the 
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ideal case. However, in many practical situations the positions and the polarities of the peaks 
are not so well defined. Video processing causes the correlation peaks to be smeared. Peaks 
are broadened, so that the central position is no longer well-defined. Even the sign of a peak 
may be incorrectly retrieved due to large negative side lobes. This is illustrated in Fig. 7, 
5 which shows the correlation functions imder sub-ideal conditions. Peaks 33 and 34 have now 
been detected, at different positions. This phenomenon is referred to as peak jitter. Peak 34 
has also the wrong polarity, so that the detector cannot determine the direction of the shift 
vector. Needless to say that the vector k' in Fig. 7 may lead to erroneous detection of the 
payload K. 

1 0 The shift vector retrieving circuit 28 (Fig. 4) is arranged to correctly retrieve 

the shift vector k, even if the peaks are smeared. The operation of one embodiment of this 
circuit will now be described with reference to Fig. 8, which shows a flow chart of 
operational steps: 

Step 51 : Detennine the position pi of the largest value in abs(B), which is larger than a 
1 5 predetermined threshold T. Determine also the position p2 of the second largest 

value in abs(B), which does not belong to a suitably chosen neighborhood of pi. 

The exclusion of an appropriate neighborhood of pi prevents a large sub-peak of pi 

being selected as p2. The threshold T can be set in accordance with the teaching of 

International Patent Application WO-A-99/45706. 
20 Step 52: Choose a window Wi (35 in Fig. 7), which includes pi, and an equally sized 

window W2 (36 in Fig. 7), which includes p2. The window may be a rectangle or 

circle in the case of two-dimensional peak patterns. 
Step 53: Determine to which extent the correlation fimction in wj and wa resemble each 

other. To this end, the circuit computes a match value e(wi,W2). Various methods 
25 of computing match values are known in the art. A simple correlation is an 

appropriate choice. The more the waveforms within the windows resemble each 

other, the larger the absolute value of e(wi,W2) will be. The sign of e(wi,W2) 

indicates whether the waveform in window wi resembles the true or inverted 

waveform in window W2» 
30 Step 54: Repeat step 53 for all possible positions of windows wi and/or W2. Note that a 

cyclic shift has been applied by the embedder. A window exceeding one of the 

ends of the bufier B must be wrapped around. 
Step 55: Determine the largest absolute value of the match value e(wi,W2) and its sign. 



10 
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Step 56: Determine the relative distance between the windows for which abs(e(wi,W2)) is 
maximal. In Fig, 7, said windows are denoted 37 and 38, respectively. The thus 
determined distance is the vector k which is looked for. 

The method described above has some drawbacks. It is computationally 
expensive, because all possible pairs of windows are evaluated. Furthermore, the geometry 
and size of the windows are fixed. If the windows are too large, much of the matching is done 
with samples of the correlation buffer B which are not sub-peaks, but noise samples. If the 
windows are too small and the peaks are broad, the wrong vector can be extracted. Fig. 9 
shows a flow chart of operational steps carried out by another embodiment of the vector 
retrieving circuit. The operation will now be described with reference to Fig, 10, which 
shows a top view of two-dimensional spatial correlation functions. 

Step 61 : Determine two peaks pi and p2 in buffer B. This step is similar to step 51 in Fig. 8. 
St^ 62: Choose a window wj^ which consists of local large peaks (larger than a threshold 

T) in the proximity of pi. In Fig. 10, local peaks are shown as solid pixels. 
15 Step 63: Calculate the vector v=p2-pi. This vector (71 in Fig. 10) can be considered a first 

estimate for the vector k looked for. 
Step 64: Determine for the vector v the match value e(wi,W2) between the correlation 

patterns in W| and W2, where W2=wi+v (72 in Fig. 10). This step is extremely 

efficient if the correlation noise for non-peaks in wj (the white rectangles in 
20 Fig. 1 0) is set to zero. 

Step 65: Repeat step 64 for all possible vectors in a proximity of v. In this example, the 

proximity is a square area 77 of 1 1x1 1 pixels around v. One such vector in the 

proximity of v is denoted 73 and the corresponding window W2 is denoted 74 in 

Fig. 10. 

25 Step 66: Determine the largest absolute value of the match value e(wj ,W2) and its sign- In 
Fig. 10, the vector and window W2 for which the match value is maximal are 
denoted 75 and 76, respectively. The local peaks in window 76 (shown as solid 
rectangles) appear to be the best match for the local peaks m window wi . 
Step 67: Derive the embedded payload K from the vector k and the sign which are 
30 associated to said largest value. 

It should further be noted that the payload data may also be represented by the 
relative polarities of multiple watermarks only. In that case, the detector needs not determine 
the distance between the most resembling windows wj and W2, but merely the sign of the 
match value e(wi,W2). 
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The methods described above look for clusters of peaks and determine the 
optimal translation to match these clusters. More precisely, having determined a first and 
second cluster of peaks, a vector v is searched such that the first cluster shifted over v 
matches the second cluster or its inverse in the best possible way. The outcome of such a 
matching is a vector v and a sign s, the sign indicating whether or not the clusters have the 
same or opposite polarity. It has been foimd that it is difficult to assign an absolute sign to a 
cluster of peaks. The encoding of payload should therefore better not rely on absolute signs 
(vectors k and -k should preferably represent the same payload data). 

The invention can be simmiarized as follows. Data is hidden in an information 
signal by encoding payload in relative positions and/or polarities of multiple (possibly equal) 
embedded watermarks. The payload is retrieved by computing the correlation between the 
watennark(s) and the signal as a function of the position. The distance between peaks of the 
correlation function represents the payload. In order to precisely detect said distance, even if 
the peaks are smeared, the correlation fimction in a window (wi) around one peak (pi) is 
compared with the correlation function in a similar v^dndow (wa) around another peak (p2). 
This is done for different positions of said windows. The distance (k) between the v\dndows 
(37,38) for which the correlation functions most resemble each other is the distance 
representing the payload. 
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CLAIMS: 



1 - A method of detecting payload data encoded into the relative positions of 

watennarks embedded in an infomiation signal, comprising the steps of: 

- computing (23-27) the coirelations between each watemiark and said signal as a fiinction 
of the position of said applied watennark with respect to the signal; 

5 - determining (28) the relative positions of peaks (3 1,32) of said correlation fimctions; and 

- detecting (29) said payload data from said relative positions of said peaks; 
characterized in that said step (28) of determining the relative positions of peaks comprises: 
~ computing (53;64) the resemblance of the correlation function in a window (wi) around 

one of said peaks (pi) to the correlation function in a window (W2) around another one 
10 (P2) of said peaks, for a plurality of positions of at least one of said windows; and 

- determining (55; 66) the positions of said windows for which said resemblance has an 
extreme value. 

2. A method of detecting payload data encoded into the relative polarities of 
1 5 watermarks embedded in an information signal, comprising the steps of: 

- computing (23-27) the correlation between the watermarks and said sig;nal as a function 
of the position of said watermark with respect to the signal; 

- determining (28) the relative polarities of peaks (3 1 ,32) of said correlation function; and 

- detecting (29) said payload data firom said relative polarities of said peaks; 

20 characterized in that said step (28) of determining the relative polarities of peaks comprises: 

- computing (53;64) the resemblance of the correlation function in a window (wi) around 
one of said peaks (pi) to the correlation function in a window (W2) around another one 
(P2) of said peaks; and 

- determining (55; 66) the sign of said resemblance value. 

25 

3. A method as claimed in claim 1 or 2, wherein the correlation fimction in a 
window around a peak is represented by the correlation values of local peaks in said window. 
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4. An arrangement for detecting payload data encoded into the relative positions 
of watermarks embedded in an information signal, comprising: 

- means (23-27) for computing the correlations between each watermark and said signal as 
a fimction of the position of said applied watermark with respect to the signal; 

5 - means (28) for determining the relative positions of peaks (3 1 ,32) of said correlation 
functions; and 

- means (29) for detecting said payload data fix>m said relative positions of said peaks; 
characterized in that said means (28) for determining the relative positions of peaks 

. comprises: 

10 - means for computing (53;64) the resemblance of the correlation function in a window 
(wi) aroimd one of said peaks (pi) to the correlation function in a window (W2) around 
another one (P2) of said peaks, for a plurality of positions of at least one of said windows; 
and 

- means for determining (55; 66) the positions of said windows for which said resemblance 
1 5 has an extreme value. 

5. An arrangement for detecting payload data encoded into the relative polarities 
of watermarks embedded in an information signal, comprising: 

- means (23-27) for computing the correlation between the watermarks and said signal as a 
20 function of the position of said watermark with respect to the signal; 

- means (28) for determining the relative polarities of peaks (3 1 ,32) of said correlation 
function; and 

- means (29) for detecting said payload data from said relative polarities of said peaks; 
characterized in that said means (28) for determining the relative polarities of peaks 

25 comprises: 

- means (53;64) for computing the resemblance ofthe correlation function in a window 
(wi) around one of said peaks (pi) to the correlation function in a window (W2) around 
another one (pi) of said peaks; 

means (55; 66) for determining the sign of said resemblance value. 

30 

6. An arrangement as claimed in claim 4 or 5, wherein the correlation fimction in 
a window around a peak is represented by the correlation values of local peaks in said 
window. 
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